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Lean  Startup  Principles 


Entrepreneurs  are  everywhere 

Entrepreneurship  is  management 

Validated  Learning 

Build  -  Measure  -  Learn 


Innovation  Accounting 


What  is  a  startup? 

A  startup  is  a  human  institution  designed  to 
deliver  a  new  product  or  service  under 
conditions  of  extreme  uncertainty. 


Nothing  to  do  with  size  of  company,  sector  of  the 
economy,  or  industry 


What  is  a  startup? 


STARTUP 


EXPERIMENT 


Who  to  Blame 


Father  of  scientific 
management 

Study  work  to  find  the  best 
way 

Management  by  exception 

Standardize  work  into  tasks 

Compensate  workers  based 
on  performance 


Frederick  Winslow  Taylor 
(1856-1915) 


"In  the  past,  the  man  was 
first.  In  the  future,  the 
system  will  be  first." 
(1911) 


Entrepreneurship  is  management 


Our  goal  is  to  create  an  institution,  not  just  a 
product 

Traditional  management  practices  fail 

-  "general  management"  as  taught  to  MBAs 

Need  practices  and  principles  geared  to  the 
startup  context  of  extreme  uncertainty 

Not  just  for  "two  guys  in  a  garage" 


DoHH^7 


"I'm  not  leaving  you,  I'm  pivoting  to  another  man." 


The  Pivot 


What  do  successful  startups  have  in  common? 

-  They  started  out  as  digital  cash  for  PDAs,  but  evolved  into  online 
payments  for  eBay. 

-  They  started  building  BASIC  interpreters,  but  evolved 
into  the  world's  largest  operating  systems  monopoly. 

-  They  were  shocked  to  discover  their  online  games 
company  was  actually  a  photo-sharing  site. 


Pivot:  change  directions  but  stay  grounded  in  what 
we've  learned. 


Speed  Wins 


If  we  can  reduce  the  time  between  pivots 


We  can  increase  our  odds  of  success 


Before  we  run  out  of  money 


Achieving  Failure 

If  we're  building  something  nobody  wants,  what 
does  it  matter  if  we  accomplish  it: 


On  time? 

On  budget? 

With  high  quality? 

With  beautiful  design? 


Achieving  Failure  =  successfully  executing  a  bad 


Lean  Startup  Principles 


Entrepreneurs  are  everywhere 

Entrepreneurship  is  management 

Validated  Learning 

Build  -  Measure  -  Learn 


There's  much  more... 


IDEAS 


Learn  Faster 


Split  Tests 

Customer  Development 

Five  Whys 

Customer  Advisory  Board 

Falsifiable  Hypotheses 

Product  Owner 

Accountability 

Customer  Archetypes 

Cross-functional  Teams 

Semi-autonomous  Teams 

Smoke  Tests 


LEARN 


Measure  Faster 


Split  Tests 

Continuous  Deployment 

Usability  Tests 

Real-time  Monitoring  &  Alerting 

Customer  Liaison 


Measure  Faster 

Funnel  Analysis 

Cohort  Analysis 

Net  Promoter  Score 

Search  Engine  Marketing 

Predictive  Monitoring 


Build  Faster 


Unit  Tests 

Usability  Tests 

Continuous  Integration 

Incremental  Deployment 

Free  &  Open-Source 

Cloud  Computing 

Cluster  Immune  System 

Just-in-time  Scalability 

Refactoring 

Developer  Sandbox 

Minimum  Viable  Product 


Lean  Startup  Principles 


Entrepreneurs  are  everywhere 

Entrepreneurship  is  management 

Validated  Learning 

Build  -  Measure  -  Learn 


Innovation  Accounting 


Innovation  Accounting 

The  Three  Learning  Milestones 


1 .  Establish  the  baseline 

Build  a  Minimum  Viable  Product  (MVP) 
Measure  how  customers  behave  right  now 


Tune  the  engine 

-  Experiment  to  see  if  we  can  improve  metrics  from  the 
baseline  towards  the  ideal 


Pivot  or  persevere 

-  When  experiments  reach  diminishing  returns,  it's  time  to 
pivot. 


Questions 


How  do  we  know  when  to  pivot? 

Vision  or  Strategy  or  Product? 

What  should  we  measure? 

How  do  products  grow? 

Are  we  creating  value? 

What's  in  the  MVP? 

Can  we  go  faster? 


Thanks! 


Buy  the  book  @ 

http://theleanstartup.com/book 

Blog:  http://StartupLessonsLearned.com 
Get  in  touch  (#leanstartup) 

-  http://twitter.com/ericries 

-  eric@theleanstartup.com 

Additional  resources 

-  http://theleanstartup.com 

-  Lean  Startup  Wiki: 
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Myth  #1 

Myth 

Lean  means  cheap.  Lean  startups  try  to 
spend  as  little  money  as  possible. 


Truth 
The  Lean  Startup  method  is  not  about  cost, 

it  is  about  speed 


Myth  #2 

Myth 

The  Lean  Startup  is  only  for 
Web  2. 0/internet/consumer  software  companies 


Truth 
The  Lean  Startup  applies  to  all  companies  that 
face  uncertainty  about  what  customers  will  want, 


Myth  #3 


Myth 


Lean  Startups  are  small  bootstrapped  startups 


Truth 
Lean  Startups  are  ambitious  and  are  able 

to  deploy  large  amounts  of  capital. 


Myth  #4 


Myth 

Lean  Startups  replace  vision  with  data 

or  customer  feedback. 


Truth 
Lean  Startups  are  driven  by  a  compelling  vision, 
and  are  rigorous  about  testing  each  element  of 

this  vision 


Lean  Startup  Principles 


Entrepreneurs  are  everywhere 

Entrepreneurship  is  management 

Validated  Learning 

Build  -  Measure  -  Learn 


Innovation  Accounting 


Minimum  Viable  Product 


Visionary  customers  can  "fill  in  the  gaps"  on 
missing  features,  if  the  product  solves  a  real 
problem 

Allows  us  to  achieve  a  big  vision  in  small 
increments  without  going  in  circles 

Requires  a  commitment  to  iteration 

MVP  is  only  for  BIG  VISION  products; 
unnecessary  for  minimal  products. 


Continuous  Deployment 


Learn  Faster 


Customer  Development 
Five  Whys 


IDEAS 


LEARN 


Build  Faster 


Continuous  Deployment 

Small  Batches 

Minimum  Viable  Product 

Refactoring 


CODE 


Measure  Faster 


MEASURE 


Split  Testing 
Actionable  Metrics 
Net  Promoter  Score 


SEM 


Continuous  Deployment  Principles 


Have  every  problem  once 


Stop  the  line  when  anything  fails 


Fast  response  over  prevention 


Continuous  Deployment 


Deploy  new  software  quickly 

-  At  IMVU  time  from  check-in  to  production  =  20  minutes 

Tell  a  good  change  from  a  bad  change  (quickly) 
Revert  a  bad  change  quickly 

-  And  "shut  down  the  line" 

Work  in  small  batches 

-  At  IMVU,  a  large  batch  =  3  days  worth  of  work 

Break  large  projects  down  into  small  batches 


Cluster  Immune  System 

What  it  looks  like  to  ship  one  piece  of  code  to  production. 


Run  tests  locally  (SimpleTest,  Selenium) 

-  Everyone  has  a  complete  sandbox 

Continuous  Integration  Server  (BuildBot) 

-  All  tests  must  pass  or  "shut  down  the  line" 

-  Automatic  feedback  if  the  team  is  going  too  fast 

Incremental  deploy 

-  Monitor  cluster  and  business  metrics  in  real-time 

-  Reject  changes  that  move  metrics  out-of-bounds 

Alerting  &  Predictive  monitoring  (Nagios) 

-  Monitor  all  metrics  that  stakeholders  care  about 

-  If  any  metric  goes  out-of-bounds,  wake  somebody  up 

-  Use  historical  trends  to  predict  acceptable  bounds 

When  customers  see  a  failure 

-  Fix  the  problem  for  customers 
Improve  your  defenses  at  each  level 


Minimum  Viable  Product 


Learn  Faster 


Customer  Development 
Five  Whys 


IDEAS 


LEARN 


Build  Faster 


Continuous  Deployment 

Small  Batches 

Minimum  Viable  Product 

Refactoring 


CODE 


Measure  Faster 


MEASURE 


Split  Testing 
Actionable  Metrics 
Net  Promoter  Score 


SEM 


Why  do  we  build  products? 


Delight  customers 

Get  lots  of  them  signed  up 

Make  a  lot  of  money 

Realize  a  big  vision;  change  the  world 

Learn  to  predict  the  future 


Possible  Approaches 


"Maximize  chances  of  success" 

-  build  a  great  product  with  enough  features  that  increase  the 
odds  that  customers  will  want  it 

-  Problem:  no  feedback  until  the  end,  might  be  too  late  to  adjust 


"Release  early,  release  often" 

-  Get  as  much  feedback  as  possible,  as  soon  as  possible 

-  Problem:  run  around  in  circles,  chasing  what  customers  think 
they  want 


Minimum  Viable  Product 


The  minimum  set  of  features  needed  to  learn  from 
earlyvangelists  -  visionary  early  adopters 

-  Avoid  building  products  that  nobody  wants 

-  Maximize  the  learning  per  dollar  spent 


Probably  much  more  minimum  than  you  think! 


Minimum  Viable  Product 


Visionary  customers  can  "fill  in  the  gaps"  on 
missing  features,  if  the  product  solves  a  real 
problem 

Allows  us  to  achieve  a  big  vision  in  small 
increments  without  going  in  circles 

Requires  a  commitment  to  iteration 

MVP  is  only  for  BIG  VISION  products; 
unnecessary  for  minimal  products. 


Techniques 


Smoke  testing  with  landing  pages,  AdWords 

SEM  on  five  dollars  a  day 

In-product  split  testing 

Paper  prototypes 

Customer  discovery/validation 

Removing  features  ("cut  and  paste") 


Fears 


False  negative:  "customers  would  have  liked  the 
full  product,  but  the  MVP  sucks,  so  we 
abandoned  the  vision" 

Visionary  complex:  "but  customers  don't  know 
what  they  want!" 


Too  busy  to  learn:  "it  would  be  faster  to  just  build 
it  right,  all  this  measuring  distracts  from 
delighting  customers" 


Five  Whys 


Learn  Faster 


Five  Whys  Root 
Cause  Analysis 


LEARN 


BUILD 


Continuous  Deployment 


CODE 


MEASURE 


Measure  Faster 


Rapid  Split  Tests 


Five  Whys  Root  Cause  Analysis 


A  technique  for  continuous  improvement  of 
company  process. 

Ask  "why"  five  times  when  something 
unexpected  happens. 

Make  proportional  investments  in  prevention  at 
all  five  levels  of  the  hierarchy. 

Behind  every  supposed  technical  problem  is 
usually  a  human  problem.  Fix  the  cause,  not  just 
the  symptom. 


Rapid  Split  Tests 


Learn  Faster 


Five  Whys  Root 
Cause  Analysis 


LEARN 


BUILD 


Continuous  Deployment 


CODE 


MEASURE 


Measure  Faster 


Rapid  Split  Tests 


Split-testing  all  the  time 


A/B  testing  is  key  to  validating  your  hypotheses 

Has  to  be  simple  enough  for  everyone  to  use 
and  understand  it 

Make  creating  a  split-test  no  more  than  one  line 
of  code: 


if(  setup_experiment(...)  ==  "control" )  { 

//  do  it  the  old  way 
}  else  { 

//  do  it  the  new  way 


Measure  the  Macro 


Always  look  at  cohort-based  metrics  over  time 
Split-test  the  small,  measure  the  large 


Control  Group  (A) 

Experiment  (B) 

#  Registered 

1025 

1099 

Downloads 

755  (73%) 

733  (67%) 

Active  days  0-1 

600  (58%)  ^^^| 

650  (59%)  ^^^H 

Active  days  1-3 

500  (48%) 

545  (49%) 

Active  days  3-10 

300  (29%)  ^^^| 

330  (30%)  ^^^H 

Active  days  10-30 

250  (24%) 

290  (26%) 

Total  Revenue 

$3210.50  ^^^| 

$3450.10   ^^^H 

RPU 

$3.13 

$3.14 
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Minimum  Viable  Product 
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